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[57] ABSTRACT 

A display screen of an image display section is divided into 
an image display area for displaying an image and an 
operation panel display area, an image (base image) imaged 
by an X-ray CT apparatus, for example, is displayed on the 
upper part (base area) of the image display area, an image 
(match image) imaged by a MRI apparatus is displayed in 
the middle part (match area), and an operation panel, which 
is composed of an operation panel display area 2 of the 
image display section and respective operation keys for 
aligning the images, is displayed thereon. When the opera- 
tion panel is operated and fit points are provided to the 
images or a region of interest (ROI) is set on the base image, 
a CPU aligns and composes the respective images based on 
the fit points or ROI, and displays the fusion image on the 
lower part (fusion area) of the image display area. As a 
result, both the images can be compared with each other 
visually. 

12 Claims, 19 Drawing Sheets 
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MEDICAL IMAGE PROCESSING In order to achieve the above object, there is provided a 

APPARATUS medical image processing apparatus comprising image cap- 

turing means for capturing a plurality of medical images 
BACKGROUND OF THE INVENTION ob(aincd by ^ m object to be examined, aligning 

1. Field of the Invention 5 means for aligning the medical images captured by the 
The present invention relates to a medical image process- capturing means, fusion image forming means for 

ing apparatus in which accuracy of diagnosis, facilitation of forming fusion image by superimposing the medical images 
a medical treatment plan, etc. are improved by displaying aligned by the aligning means on each other, display means 
images which have been obtained by medical image diag- for displaying the medical images thereon, display control 
nostic apparatuses using different image pickup methods 10 means for controlling the display of the fusion image of the 
such as an X-ray CT apparatus, MRI apparatus (magnetic respective medical image formed by the fusion image form- 
resonance imaging apparatus), nuclear medicine diagnostic mg mcans on thc display means. 

apparatus, ultrasonic image diagnostic apparatus and X-ray Examples of medical images to be composed and dis- 

diagnostic apparatus. played are two (or more) medical images imaged by an 

2. Prior Art 15 imaging apparatus using the same imaging methods, and 
Conventionally, images obtained by medical image diag- medical ima S ed b V ^fi^e apparatuses using dif- 

uostic apparatuses (modality) such as an X-ray CT apparatus ferenl ^6*6 methods such * ™ X " rav CT a PP aratus 

and MRI apparatus are two-dimensionally coaxial MRI a PP aratus - 

tomograms, so it is difficult to obtain and diagnose a When such medical images are captured, the aligning 

three-dimensionally inside structure of a human body from 20 means aligns each medical image, and the fusion image 

one coaxial tomogram. Therefore, only one coaxial tomo- forming means superimposes the aligned medical images on 

gram is imaged infrequently, and in general, a plurality eacn other so as to form a fusion image. The display control 

coaxial tomograms including an internal organs of interest means controls the display of the fusion image of the 

(internal organs to be diagnosed) are imaged. medical images formed by the fusion image forming means 

According to the improvement in the image pick-up on me msplay means, 

technique in recent years, a plurality of coaxial tomograms As a result, an operator can compare the images with each 

can be picked up at high speed in the X-ray CT apparatus other realistically and visually based on the fusion image 

and MRI apparatus employing a helical scanning method or displayed on the display means, and can make a diagnosis 

SPECT apparatus (Single Photon Emission CT: nuclear 3Q and plan medical treatment easily in the clinical examina- 

medicine image CT apparatus), etc., and in a clinical tion. 

examination, a diagnosis is made and medical treatment is In the preferable embodiment of the present invention, the 

planned based on a plurality of the coaxial tomograms display control means controls the display of the medical 

imaged by these modalities. images captured by the image capturing means as well as the 

In the field of the clinical examinations, a diagnosis is 35 fusion image on the display means, 

made and medical treatment is planned mostly by comparing For a fuller understanding of the nature and advantages of 

different tomograms which were imaged in different times, the invention, reference should be made to the ensuing 

but even in the case of the same patient, it is difficult to detailed description taken in conjunction with the aceom- 

image each coaxial tomogram in tbe same pose unless a panying drawings, 

special locking instrument is attached to tbe patient. 40 

Moreover, it is also difficult to adjust each imaging condi- BRIEF DESCRIPTION OF THE DRAWINGS 
tions such as a slice thickness, slice pitch and image size per 

modality and then to image each coaxial tomogram. In me accompanying drawings: 

For this reason, in the case where coaxial tomograms FIG - 1 is a schematic drawing showing a result of 

imaged by modalities using different imaging methods are 45 composing three-dimensionally surface images obtained by 

compared with each other, a doctor has superimposed the different modalities with each other and displaying fusion 

coaxial tomograms on each other imaginarily based on his images on an image display section provided to a medical 

experiences and knowledge, and has made a diagnosis and ^age processing apparatus according to embodiment of the 

planned medical treatment according to the invisible and present invention; 

imaginary fusion coaxial tomogram. 50 FIG. 2 is a drawing showing an operation panel as well as 

Then, because of such inconvenience, a medical image mc ^ ion images displayed on the image display section; 

processing apparatus, in which coaxial tomograms imaged FIG. 3 is a block diagram of the medical image processing 

by modalities using different imaging methods such as a apparatus according to the embodiment of the present inven- 

co axial tomogram imaged by the X-ray CT apparatus and a tion; 

coaxial tomogram imaged by the MRI apparatus can be 55 p [G 4 ^ a drawing showing a control area of the 

compared with each other realistically and visually, is operation panel; 

strongly desired to be developed. FIG. 5 is a drawing explaining display image selecting 

SUMMARY OF THE INVENTION keys for specifying a display direction of images displayed 

. . . . j . . - on the control area: 

The present invention is invented in view of the 60 ' 

circumstances, and it is an object of the present invention to FIG 6 is a drawing explaining the display directions of 9 

provide a strongly desired medical image processing appa- sections of the images displayed on the image display 

ratus where images of the same position with the same size, section variable by the display image selecting key; 

which have been imaged by modalities using different FIGS. 7A and 7B are drawings showing a rendering area 

imaging methods, are superimposed on and composed with 65 of the operation panel; 

each other and are displayed so as to be able to be compared FIG. 8 is a drawing explaining an image display area 

with each other realistically and visually. which can be controlled by dragging with a mouse; 
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FIG. 9 is a drawing explaining a relationship between an display area 2, the image display area 1 is divided into three 

operation in a mouse mode area of the operation panel and sections, namely, an upper part, a middle part and a lower 

the display panel; part. An image imaged by an X-ray CT apparatus, for 

FIG. 10 is a drawing explaining setting of a window width example, is displayed on the upper part as a base area 3, an 

and a window level of the display screen in the mouse mode 5 imaged by an MM apparatus, for cxample^is^dis- 

area . played on the middle part aVa match area 4, and a fusion 

FIG. 11 is a drawing explaining fit mode specifying keys ™& obtained b J <r°»P°^f & ^S 65 « ° n 

displayed on the mouse mode area when fit points for lower P art 85 a fasion area 5 - 

aligning images of different modalities are specified; A plurality of operation keys relating to display of such a 

HO. 12 is a drawing explaining a displayed direction 10 ima 8 e » ***** 00 °P era,ion P anel 

changing key displayed in the mouse mode area when a area ' 

display direction of the pick-up images displayed on the More specifically, as shown in FIG. 2, the operation panel 

image display section is changed; display area 2 is composed of a control area 6, a rendering 

FIG. 13 is a drawing explaining the display direction 15 arca 7 > a m0l f ft e mode # area 8, a view mode area 9, a fusion 

changing keys displayed in the mouse mode area when a bajance arca 10 > a P 0SltlOD n > a savc 12 aad a S nd 

display position of the images displayed on the image arca 

display section is changed; As shown in FIG. 4, the control area 6 is provided with a 

FIG. 14 is a drawing explaining zoom rate changing keys ^P 1 "? specifying key 30 for selecting a display format 

displayed in the mouse mode area when a zoom rate of the 20 0™-«*tion displays-section display) of an image dis- 

images displayed on the image display section is changed; m image display area a study selecting key 33, 

~z . _\ . . , . . . , etc. for selecting an image (study) to be displayed m the 

FIGS. 15A and 15B are drawings explaining a view mode ima ^mfcy area> 

area of the operation panel; . , . m ' » , -T> , 

„ - . , . , . o... * . As shown in FIGS. 7A and 7B, the rendering area 7 is 

nG.16isadrawmgshowmgaftisionbalanceareaofthe ^ provided ^ a display ^ode switching key 41, etc. for 

operation panel, switching an image to be displayed in the image display area 

FIG. 17 is a drawing showing a position area of the i between a coaxial tomogram (MPItyand a surface image, 

operation panel; As shown in FIG. 10, the mouse mode-area* is provided 

FIG. 18 is a drawing showing a grid area of the operation mouse mode s^lectingjkeys 43 for selecting respective 

panel; 30 modes, mentioned laterr and a display panel^49 to, be 

FIG. 19 is a drawing explaining a change of a position of displayed.acx:ording-to-a mode selected by me^mouse mode 

a cross-sectional cursor in the grid area; selecting keys .343 

FIG. 20 is a drawing showing a setting screen of the fit As the rr^use-mode-selectmgEkeys 43, a bright mode 

points displayed on the operation panel when the fit mode specifying key 44 shown in FIG. 10 for^specifying a bright 

specifying key is clicked; 35 mode adjusting a window width and a window level, a fit 

FIG. 21 is a schematic drawing showing a result of mode display key 45 displayed at the lime of a point fit mode 

composing coaxial tomograms obtained by different modali- and surface fit mode shown in FIG. 11 for aligning the 

ties with each other and displaying a fusion coaxial tomo- respective images in the base area 3 and match area 4, and ; 

gram on the image display section; a display direction changing mode specifying key 46 shown 

FIG. 22 is a drawing showing a dialogue box which 40 in FIG. 12 for spectfying . a. display direction changing mode * 

displays "reliability" of the fusion image obtained by mak- J^^B"* ^P^recUons of me respective images in the 

ing alignment at the lit points (or ROl); base area 3 ««* matc $ area 4 m Prided. 

FIG. 23 is a graph showing a relationship between an A . In t addition > as u me . ^ 43 > . a 

actual shifting amount and "reliability" of the images; 45 ^ la ™f ltl0n c !f? ging ™pde specifying key 47 shown in 

n „ _ . ( , , , . , FIG. 13 for specifying a display position changing mode 

FTC. 24 is a drawing showing a dialogue box which chan ^ ^ positions of me respective images in the 

disp ays a rotation amount, a panning amount, etc. of the bage afea 3 and match afea 4 & m rate changing mode 

displayed images; specifying key 48 shown in FIG. 14 for specifying a zoom 

FIG. 25 is a drawing showing a dialogue box displayed in ra te changing mode changing zoom rates of the respective 

the mouse mode area when the images obtained by different 5 ° images in the base area 3 and match area 4 are provided, 

modalities are composed by setting region of interest (ROI); M ^ QWU {q mQS 1SA ^ 15fij me ^ modc area 9 

FIGS. 26Aand26B are drawings explaining a rectangular is provided with display direction specifying keys 69 for 

ROI and a circular ROI set by operating the dialogue box; changing display directions of the areas 3, 4 and 5. 

and 55 As shown in FIG. 16, the fusion balance area 10 is 

FIG. 27 is a drawing showing fusion images obtained by provided with a density adjusting slider 70 for adjusting a 

aligning according to the rectangular ROI. density of the display image in the fusion area 5. 

DETAILED DESCRIPTION OF THE ^& ^ owa m 17, the position area 11 is provided with 

PREFERRED EMBODIMENTS a sur ^ ace & x 7 * f° r specifying that the respective images 

60 in the base area 3 and match area 4 are aligned by setting a 

The following describes preferable embodiments of the region of interest (ROI) ; a point fit key 72 for specifying that 

medical image processing apparatus of the present invention the-respective images in the areas 3 and 4 are aligned by 

in detail on reference to the drawings. ^ setting fit points, and a set key 73 for specifying display of 

First, in the medical image processing apparatus accord- a dialogue box (FIG. 24) for displaying a rotation amount 

ing to embodiment of the present invention, as shown in 65 and panning amount calculated on a basis of an original 

FIG. 1, a display screen of an~image-display section is display direction and display position of the image currently 

divided into an image display area 1 and an operation panel displayed in the match area. 
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The save area 12 is provided with a save key for sped- coaxial tomogram data, and a three^mensibrial pseudo 

fying recording of the image currently displayed in the image is formed based on the coaxial tomogram data for 

match area 4. plural pieces in the image processing section 22, mentioned 

As shown in FIG. 18, the grid area 13 is provided with a later, so as to be displayed on the image display section 21. 

cross-sectional cursor display/non-display selecting key for * Next, when respective coaxial tomogram data from the 

specifying display/non-display, etc. of a cross-sectional cur- modalities using different imaging methods are stored into 

sor to be displayed on a coaxial tomogram (MPR) when the the three-dimensional image data base 26, an operator 

coaxial tomogram is selected as a mode of an image to be operates the pointing apparatus 24 so as to select ID or (and) 

displayed in the image display area in the rendering area 7 name of a patient to be diagnosed according to the com- 

(FIG. 7A), a cursor r mpving condition switching key 75 for *o parison of the data, and a modality, 

selecting as to whether or not the cross^sectional cursors After input, the CPU 27 compares IDs or (and) names of 

displayed in the^base area 3 and match area 4 are synchro- two patients inputted per modality with each other, and when 

nized with each other and changed, a cursor position chang- both coincide with each other, the CPU 27 waits for 

ing keys 77 through 79, etc. for specifying movements of the reading of the respective image data, 

cross-sectional cursors. , 15 In addition, when the both do not coincide with each 

An electrical configuration of such a medical image other> the cpu 27 controls the image display section 21 so 

processing apparatus is shown in FIG. 3, and it has an image ^ a warning meS sage such that "the patient's IDs (or 

display section 21 composed of a monitor unit fo^tapjaying names) do not Continue the process? YES NO" is 

1111 a ^ es ' uoq <P^^l>* e te.^ ^e^above'areas v a driver; displayed. Then, when "NO" on the display screen is clicked 

etc., -an image processing section 22 for forming a three-: *> by me pomtmg unit 24, input of a patent's ID, etc. is required 

dimensional pseudo image based on -a- plurality of two-j again Moreove r, when "YES" on the display screen is 

dimensional images imaged by the modalities such as the clicked> ^ cpv 27 waits for reading of the image data. 

X-ray CT apparatus and MRI apparatus so as to display the A l4 u . . A , . 

, u J ■ . , . . j. , \. J As a result, it is previously prevented by comparing 

three-dimensional image on the image display section 2o . , . ' e*..* 4 5 * ».« » .* .« . 

* . .- & ~*c'/ T ■ 1 c *i_ is coaxial tomograms of different patients with each other that 

and an image storage section 23 for storing a plurality of the a . 0 . it _ A r .. . . , , A . 

j. « . j, 4 . °j . . inconvenience such that a wrong diagnosis is made, and thus 

two-dimensional images imaged by the modalities thereinto - Cl e j- • iT *«. j- 1 • 

u *u *u j- ■ ■ c j accuracy and safety of a diagnosis by the medical image 

when the mree-dimensional image is formed. r J , . * & 

. . -. . processing apparatus can be obtamed. 

In addition, the medical image processing apparatus has a , „ „ . ✓ , 

pointing device 24 such as- a mouse and keyboard for Here '. lhe <™»P™ of the images of different IDs (the 

operating the operation panel displayed in the operation 30 <»mpansonof the images of different patients) ^ effective in 

panel display area 2, an image input interface 25 for input- ° b *f ™°g a difference in diagnose portions bebyeen a 

ting and outputting a plurality of the two-dimensional bealth y ?*** nt ^ d ? P atlent m bad condlhon - For *" 

images imaged by the' modalities such as the X-ray CT «f^ «i the medical image processing apparatus, even if 

apparatus and MRI apparatus, a three-dimensional image ^ patient s IDs do not coincide wUh each otto, when 

daia base 26 for storing a plurality of the two-dimensional 35 execu,lon command 18 executed < ^ 15 chcked >' lhe 

images, etc. imaged by the modalities thereinto, and a process is con nue . 

central control unit (CPU) 27 for controlling the whole of the Next > wnen the coaxial tomogram data of the respective 

medical image processing apparatus. modalities are stored into the three-dimensional image data 

The respective sections 21 through 26 are connected to base 26 > me °P erator selects a format 

the CPU 27 via a bus line 28, and the CPU 27 communicates More specifically, as shown in FIG. 4, the control area 6 

with the respective sections 21 through 26 via the bus line m the operation panel display area 2 is provided with a 

28 and at the same time controls the whole of the medical display specifying key 30 composed of a one-section display 

image processing apparatus. tey 31 and a 9-section display key 32. When one desired 

The following describes an operation of the medical 45 image is desired to be displayed in me image display area 1, 

image processing apparatus according to the present the one -section display key 31 is clicked by the mouse of the 

embodiment having the above arrangement. pointing unit 24, and a study selecting key 33 is clicked so 

First, image data as comparative objects are captured from mal ^ for °«-sectiDn display is specified in the base 

the modalities. The image data are captured through the area 3 > match area 4 or 6181011 5 * 

image input interface shown in FIG. 3, and when coaxial 50 In addition, when images of three poses by the respective 

tomogram data are supplied from the different modalities modalities are desired to be displayed in the base area 3 and 

such as the X-ray CT apparatus and MRI apparatus via the match area 4 of the image display area 1, and a fusion image 

image input interface 25, the CPU 27 controls the three- 0I> me images of the respective poses by the respective 

dimensional image data base 26 so that the three- modalities is desired to be displayed in the fusion area 5, the 

dimensional image data base 26 stores the coaxial tomogram 55 9-section display key 31 is clicked by the mouse, 

data respectively into storage regions which have been The study selecting key 33 is composed of a base image 

allocated for each unit. selecting key (Base) 34 for specifying one-section display of 

Header data such as the name of an imaging apparatus an image (base image) in the base area 3, a match image 

(name of modality), the identification number of a patient selecting key (Match) 35 for specifying one-section display 

(patient's ID), the name of a patient, an image No., an eo of an image (match image) in the match area 4, and a fusion 

imaged date, an imaged time, an imaged position, the date image selecting key (Fusion) 36 for specifying one-section 

of patient's birth, sex, an imaged pose and an imaged portion display of an image (fusion image) in the fusion area 5. 

arc added to the coaxial tomogram data, and when a When one-section display is specified, the CPU 27 controls 

patient's ID and name, etc. displayed in the list are selected, the display of one-section image corresponding to clicked 

the image can be specified and read out. 65 one ot? me selecting keys 34 through 36. 

In addition, in the medical image processing apparatus, In other words, the CPU 27 detects that the one-section 

coaxial tomogram data for plural pieces are captured as the display key 31 is clicked, and controls display of a display 
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format selecting key 37 in the lower part of the one-section 
display key 31 and 9-section display key 32 as shown in 
FIG. 5. 

Hie display format selecting key 37 is composed of 
totally three keys, i.e., first through third display format 5 
specifying keys (A through C) 38 through 40. When the first 
display format specifying key (A) 38 is clicked in an MPR 
display mode for displaying a coaxial tomogram, the CPU 
27 controls display of a coronal image of the image selected 
by the study selecting key 33, and when the second display 10 
format specifying key (B) 39 is clicked in the MPR display 
mode, the CPU 27 controls display of a sagital image of the 
image selected by the study selecting key 33, and when the 
third display format specifying key (C) 40 is clicked in the 
MPR display mode, the CPU 27 controls display of an axial 15 
image of the image selected by the study selecting key 33. 

In addition, when the first display format specifying key 
(A) 38 is clicked in a surface display mode for display a 
surface image, the CPU 27 controls display of an image in 
a front direction or rear direction of the image selected by 20 
the study selecting key 33, and when the second display 
format specifying key (B) 39 is clicked in the surface display 
mode, the CPU 27 controls display of an image in a right 
direction or left direction of the image selected by the study 
selecting key 33, and when the third display format sped- 25 
fying key (C) 40 is clicked in the surface display mode, the 
CPU 27 controls display of an image in a head-to-foot 
direction or foot-to-head direction of the image selected by 
the study selecting key 33. 

Meanwhile, when the 9-section display key 32 is clicked, 
the CPU 27 detects it, and controls display of the display 
format selecting key 37 in the lower part of the one-section 
display key 31 and 9-section display key 32 in the similar 
manner to the aforementioned case where the one-section 35 
display key 31 is clicked. 

When the display format selecting key 37 is displayed, the 
operator clicks the display format specifying keys (A 
through C) 38 through 40 so as to select a display format of 
the image to be displayed in the base area 3, match area 4 ^ 
and fusion area 5. 

As shown in FIG. 6, when the first display format speci- 
fying key (A) 38 is clicked in the MPR display mode, the 
CPU 27 controls display of the coronal images of the base 
image in the base area 3, the match image in the match area 45 
4 and the fusion image in the fusion area 5 in a left file (3<z, 
4a and 5a) in each area 3 through 4. When the second 
display format specifying key (B) 39 is clicked in the MPR 
display mode, the CPU 27 controls display of sagital images 
of the base image, match image and fusion image in a middle 50 
file (3b, 4b and 5b) in each area 3 through 4. When the third 
display format specifying key (C) 40 is clicked in the MPR 
display mode, the CPU 27 controls display of axial images 
of the base image, match image and fusion image in a right 
file (3c, 4c and 5c) in each area 3 through 4. 55 

In addition, when the first display format specifying key 

(A) 38 is clicked in the surface display mode for displaying 
a surface image, the CPU 27 controls display of images in 
the front or rear direction of the base image, match image 
and fusion image in the left file (3a, 4a and 5a) in each area 60 
3 through 4. When the second display format specifying key 

(B) 39 is clicked in the surface display mode, the CPU 27 
controls display of images in right or left direction of the 
base image, match image and fusion image in the middle file 
(36, 4b and Sb) in each area 3 through 4. When the third 65 
display format key (C) 40 is clicked in the surface display 
mode, the CPU 27 controls display of images in the head- 
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to-foot or foot-to-head direction of the base image, match 
image and fusion image in the right file (3c, 4c and 5c) in 
each area 3 through 4. 

Here, when the 9-section display key 32 is clicked, a study 
cannot be selected by the study selecting key 33. 

Next, when the selection of the display format, etc. is 
completed, the operator selects the display mode of an image 
to be display in the image display area 1 using the display 
mode switching key 41 provided to the rendering area 7 
shown in FIGS. 7A and 7B in the operation panel display 
area 2. 

At the initial setting, the CPU 27 automatically selects the 
display mode (MPR display mode) of a coaxial tomogram 
(MPR) as shown in FIG. 7 A, and controls display of a word 
"MPR" on the display mode switching key 41 in the 
rendering area 7. 

In the case where the display mode initially set is switched 
to the display mode (surface display mode) of a surface 
image (Surface), the operator clicks the display mode 
switching key 41 using the mouse. After this clicking, the 
CPU 27 controls display of a word "Surface" on the display 
mode switching key 41 as shown in FIG. 7B, and at the same 
time controls display of a threshold setting key 42 
(threshold) so as to switch the display mode. 

In the medical image processing apparatus, when the 
surface image mode is selected, a three-dimensional pseudo 
image formed by a "Voxel method", mentioned later, is 
displayed. When a three-dimensional image is formed by the 
Voxel method, the threshold setting key 42 inputs a thresh- 
old value of a CT value in the case where the modality is the 
X-ray CT apparatus, for example, and inputs a threshold 
value of a proton value in the case where the modality is the 
MRI apparatus. 

When the threshold setting key 42 is clicked, the CPU 27 
forms and displays an input panel for inputting a threshold 
value. Then, an image having a CT value of not more than 
the threshold value (or not less than the threshold value) 
inputted to the input panel is formed and displayed. As a 
result, it is possible to obtain a desired surface image such 
as a surface image of only a skeleton. 

In the medical image processing apparatus, the respective 
images of the one-section display or 9-section display are 
displayed in a form of three-dimensional pseudo images 
based on a plurality of the two-dimensional coaxial tomo- 
grams by the modalities stored in the three-dimensional 
image data base 26. 

The mree-dimensional pseudo images displayed in the 
surface display mode are formed based on the "Voxel method 
by the image processing section 22 shown in FIG. 3, and 
when reading out the coaxial tomogram of the patient 
specified by the patient's ID, etc. by the modality from the 
three-dimensional image data base 26, the CPU 27 supplies 
the coaxial tomogram to the image processing section 22. 

Here, as the coaxial tomogram, in the case where the 
modality is the X-ray CT apparatus, for example, a CT value 
is read out, and in the case of the MRI apparatus, a proton 
value is read out. 

The image processing section 22 is composed of a faree-^ 
dimensional aUdressrgerierator and-a mree^iimensionally 
arithmetic high-speed processor having a high-speed image 
processing function such as a three-dimensional space filter 
and a three-dimensionally logical filter and having a high- 
speed graphics function such as a hidden-line, hidden- 
surface process, lightness calculation and projection conver- 
sion. 
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When CT values as the respective coaxial tomograms are aligning the base image with the match image, to set region 

supplied in the case where the modality is the X-ray CT of interest (ROl) for alignment, and to turn the display 

apparatus, the image processing section 22 filters the CT image, parallel transfer the display image, and change a 

values using the three-dimensional space filter so as to zoom rate of the display image. 

improve image quality as well as binarizes the CT values 5 i n the mouse mode area 8, as shown in FIG. 8, the 

according to the threshold value set by the threshold setting respective images in the base area 3 and the match area 4 can 

key 42 shown in FIG. 7. 5 e operated, and a fusion image according to the operation 

The three-dimensionally arithmetic high-speed processor in each area 3 and 4 is displayed in the fusion area 5. 

controls writing of the binarized coaxial tomograms into the In addition, on the mouse mode area 8, as shown in FIG. 

image storage section 23 which is a high-speed large- io 9 ^ mouse mode specifying keys 43> which ^ composed 

capacity memory so that the coaxial tomograms are lami- 0 f a Dt ight mode specifying key 44, a fit mode display key 

nated in the order of the image numbers (in the order of 45^ a display direction changing key 46, a display position 

slicing number). As a result, a mree-dimensional pseudo changing key 47 and a zoom rate changing mode specifying 

image is formed by the Voxel method on the image storage key 48? and a dis p i ay pane i 49 are displayed. 

^ff 11 u u j- • -r 1-- On the display panel 49, a panel having contents accord- 
After the three-dimensional image is formed on the image mg to me reS pective modes allocated to the keys 44 through 
storage section 23 in such a manner, when the display format 4g operated by the operator is displayed. Moreover, the 
37 T Dt i° ned referrin S to ™- , 5 » 0P«*fd. display panel 49 is provided with tags 49a whose number 
die CPU 27 supplies data representing the selected display co rresp onds with the number of the keys 44 through 48, 
format into the image processing section 22. 20 when ^ operator operates ^ (ags 49flj only ^ conten( of 

The three-dimensional address generator of the image the panel is switched with the mode being unchanged, 

processing section 22 calculates an address at a high speed Namel when me window M ^ kvel in the 

according to the selected display format, and transfers the base area 3 or match afea 4 afe adjusted> ^ QT dicks 

address data to the mree^imensionally ^ anthmetic high- the bright mode specifying key 44 shown in FIG. 10 in the 

speed processor at a high speed. When the. three- mQUSe mode area 8 When ^ ^ mode dfyin k 

dimensionally anthmeuc highspeed processor reads out the 44 ^ clicked> ^ cprj 2J CQn{lo]s ^ - of a 

three-dimenaonal pseudo image stored in the image storage j shown ^ FIG 1Q QQ ^ ^ { { 49 

section 23 based on the address data and performs parallel ^ ^ . J , ' . , 

processing and pipeline processing on the three-dimensional ° n me ™ dow * m \ a area selecting key 

pseudo image, the three-dimensionally arithmetic high- 30 *0 for selecting an image (area) whose window is to be 

speed processor forms the three-dimensional image of the a * UBl f* a *™ dow , Mdth .changmgkey 51, a windowwidtb 

selected display format at high speed and supplies it to the <*angmg slider 52, a window level changing key 53 a 

image display section 21. wmdow level shder H and ke y m P u * display 

A \: jt c ' ™^ m .1- _c sections 52a and 54a are displayed. 

As a result, as mentioned referring to FIG. 5, the surface , c 

image in the front or rear direction (the first display format M me toI S et m * selecting key 50, a base area selecting 

specifying key 38) selected by the display format selecting ke ? 50fl md a match 3163 Acting 506 for selecting an 

key 37, the surface image in the right or left direction (the area whose win . dow 15 t0 be adjusted are provided, and the 

second display format specifying key 39), or the surface CPU 27 rec °g Dlzes an area selected by the keys 50a and 506 

image in the head-to-foot or foot-to-head direction (the third 35 me *"» whose » 10 be adjusted, 

display format specifying key 40) can be displayed on the The window width changing key 51 adjusts window 

image display section 21. widths of the areas 3 and 4, and when a feed key 51a is 

Here, the image to be displayed in the surface display operated, the CPU 27 controls the window selected by the 

mode may be formed by, for example, a wire frame method tar e et area selecting key 50 so that the window width is 

of extracting outlines of an internal organ, etc. for the 45 widened according to the operating time. When the back 

respective coaxial tomograms as charts and composing a feed ke y Slb * operated, the CPU 27 controls the window 

surface by connecting the outlines like mesh, a surface selected by the target area selecting key 50 so that the 

method of painting over the mesh surface and displaying it window width is narrowed according to the operating time, 

with a smooth shadow, etc. besides the Voxel method. In addition, when a quick-feed key 51c is operated, the 

Next, in the medical image processing apparatus of the 50 cpu 27 co^ 1 " 015 tne window selected by the target area 
present invention, in the MPR display mode, the three- selecting key 50 so that the window width is widened 
dimensional address generator and three-dimensionally quickly according to the operating time, and when a quick- 
arithmetic high-speed processor in the image processing back feed key 51a* is operated, the CPU 27 controls the 
section 22 interpolate a density value according to a coor- window selected by the target area selecting key 50 so that 
dinate by the neighbor arithmetic based on the three- 55 me window width is narrowed quickly according to the 
dimensional pseudo image stored in the image storage operating time. 

section 23, and form a coaxial tomogram of the display The window width can be changed by the window width 

format selected by the display format selecting key 37. changing slider 52 or direct key input, and in the case where 

As a result, a coronal image, sagital image or axial image the window width is changed by the window width changing 

is formed according to the display format selected by the 60 slider 52, the operator drags the slider 52 in the right or left 

display format selecting key 37 and can be displayed on the direction using the mouse. 

image display section 21. When the slider 52 is dragged in the right direction, the 

Next, when the 9-section display is executed according to CPU 27 controls the window so that the window width is 

the arbitrary display format in such a manner, the operator widened according to the dragging amount, and when the 

operates the mouse mode area 8 so as to change a window 65 slider 52 is dragged in the left direction, the CPU 27 controls 

value of the base area 3 and match area 4 (window width and the window so that the window width is narrowed according 

window level of the display screen), to set fit points for to the dragging amount. 
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When the window width is changed by the window width clicks the display direction changing mode specifying key 

changing key 51 or window width changing slider 52, the 46 shown in FIG. 12 in the mouse mode area 8. When the 

CPU 27 controls display of a numerical value representing display direction changing mode specifying key 46 is 

the current window width on the key input display section clicked, the CPU 27 controls display of a display direction 

52a. As a result, the operator can recognize the current 5 adjusting panel shown in FIG. 12 on the display panel 49. 

window width by the numerical value. The display direction adjusting panel is provided with a 

In addition, in the case where the window width is display direction changing key 55 which is composed of a 

changed by direct key input, the operator directly writes its plane A key for changing the display direction of the left file 

numerical value into the key input display section S2a by in the image display area 1 shown in FIG. 6 (images A: 

key input. When the window width is written into the key 10 respective images in the left file in the areas 3, 4 and 5), a 

input display section 52, the CPU 27 controls display so that plane B key for changing the display direction of the middle 

the window width corresponding to the numerical value is file in the image display area 1 (images B: respective images 

obtained. in the middle file in the areas 3, 4 and 5), and a plane C key 

Next, the window level changing key 53 adjusts the for changing the display direction of the right file in the 

window levels in the areas 3 and 4, and when a feed key 53a 15 image display area 1 (images C: respective images in the 

is operated, the CPU 27 controls the window selected by the right file in the areas 3, 4 and 5). 

target area selecting key 50 so that the window level is risen When a feed key 55a of the plane A key is operated, the 

according to the operating time. When a back feed key S3b CPU 27 controls display so that the images in the left file in 

is operated, the CPU 27 controls the window selected by the the image display area 1 are rotated in the clockwise 

target area selecting key 50 so that the window level is 20 direction according to the operating time, and when a back 

lowered according to the operating time. feed key 55b is operated, the CPU 27 controls display so that 

In addition, when a quick-feed key 53c is operated, the the images in the left file in the image display area 1 are 

CPU 27 controls the window selected by the target area rotated in the counterclockwise direction according to the 

selecting key 50 so that the window level is risen quickly operating time. 

according to the operating time, and when a quick-back feed 25 In addition, when a quick-feed key 55c of the plane A key 

key 53d is operated, the CPU 27 controls the window is operated, the CPU 27 controls display so that the images 

selected by the target area selecting key 50 so that the in the left file in the image display area 1 are rotated quickly 

window level is lowered quickly according to the operating in the clockwise direction according to the operating time, 

time. and when a quick-back feed key 55d is operated, the CPU 

The window level may be changed by the window level 27 controls display so that the images in the left file in the 

changing slider 54 or direct key input, and in the case where image display area 1 are rotated quickly in the counterclock- 

the window level is changed by the window level changing wise direction according to the operating time, 

slider 54, the operator drags the slider 54 in the right or left In addition, when a feed key 56a of the plane B key is 

direction using the mouse. 35 operated, the CPU 27 controls control display so that the 

When the slider 54 is dragged in the right direction, the images in the middle file in the image display area 1 are 

CPU 27 controls display so that the window level is risen rotated in the clockwise direction according to the operating 

according to the dragging amount, and when the slider 54 is time, and when a back feed key 56b is operated, the CPU 27 

dragged in the left direction, the CPU 27 controls display so controls display so that the images in the middle file in the 

that the window level is lowered according to the dragging ^ image display area 1 are rotated in the counterclockwise 

amount direction according to the operating time. 

When the window level is changed by the window level In addition, when a quick-feed key 56c of the plane B key 

changing key 53 or window level changing slider 54 in such is operated, the CPU 27 controls display so that the images 

a manner, the CPU 27 controls display of a numerical value in the middle file in the image display area 1 are rotated 

representing the current window level on the key input 4S quickly in the clockwise direction according to the operating 

display section S4a. As a result, the operator can recognize time, and when a quick-back feed key S6d is operated, the 

the current window level by the numerical value. CPU 27 controls display so that the images in the middle file 

In addition, in the case where the window level is changed in the image display area 1 are rotated quickly in the 

by the direct key input, the operator directly writes the counterclockwise direction according to the operating time, 

numerical value into the key input display section 54a by the 50 In addition, when a feed key 57c of the plane C key is 

key input. When the window level is written into the key operated, the CPU 27 controls display so that the images in 

input display section 54c, the CPU 27 controls display so the right file in the image display area 1 are rotated in the 

that the window level corresponding to the numerical value clockwise direction according to the operating time, and 

is obtained. when a back feed key S7b is operated, the CPU 27 controls 

Next, in the medical image processing apparatus of the 55 display so that the images in the right file in the image 

present invention, the images in the base area 3 or match display area 1 are rotated in the counterclockwise direction 

area 4 can be aligned by setting fit points or ROI, the according to the operating time. 

selection of the alignment between fit points and ROI is In addition, when a quick-feed key 57c of the plane C key 

made by operating a point fit key 72 and a surface fit key 71 is operated, the CPU 27 controls display so that the images 

shown in FIG. 17. Then, when the point fit key 72 or surface 60 in the right file in the image display area 1 are rotated 

fit key 71 is operated, the CPU 27 controls display of a fit quickly in the clockwise direction according to the operating 

mode display key 45 shown in FIG. 11. For this reason, the time, and when a quick-back key 57d is operated, the CPU 

description of the fit mode display key 45 will be given later 27 controls display so that the images in the right file in the 

when the point fit key 72 or surface fit key 71 will be image display area 1 are rotated quickly in the counterclock- 

explained. 6 5 wise direction according to the operating time. 

Next, in the case where the display direction of the images Here, in the adjustment of the rotating direction, the 

in the base area 3 or match area 4 is adjusted, the operator three-dimensional address generator and three- 
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dimensionally arithmetic high-speed processor in the image tion according to the operating time, and the CPU 27 

processing section 22 calculates three-dimensional pseudo controls display so that fusion image is panned according to 

images stored in the image storage section 23 at high speed the panning of the match image. 

according to the rotating directions, and supplies the three- j n addition, when a high-speed up direction specifying 

dimensional pseudo images to the image display section 21 5 key S9g is operated, the CPU 27 controls display so that the 

so as to display the images in the respective directions. match image in the match area 3 of the plane selected by one 

Next, in the case where the display positions of the images of the selecting keys 58a through 58c is panned in the up 

in the areas 3, 4 and 5 are adjusted, the operator clicks the direction at high speed according to the operating time, and 

display position changing mode specifying key 47 shown in the CPU 27 controls display so that fusion image is panned 

FIG. 13 in the mouse mode area 8. When the display io according to the panning of the match image, 

position changing mode specifying key 47 is clicked, the In addition, when a high-speed down direction specifying 

CPU 27 controls display of a display position adjusting k ey 59^ j s operated, the CPU 27 controls display so that the 

panel shown in FIG. 13 on the display panel 49. match image in the match area 3 of the plane selected by one 

The display position adjusting panel is provided with a of the selecting keys 58a through 58c is panned in the down 

panning plane selecting key 58 for changing the display 15 direction at high speed according to the operating time, and 

positions of the images in the left, middle and right files in the CPU 27 controls display so that fusion image is panned 

the image display area 1, and a.Danning.direction specifying according to the panning, of the match image, 

key 59 for controlling panmSf of the images in the up-and- M a result> the images of the respective planes can be 

down and right-and-left directions. panned in desired positions. 

Ihe panning plane selecting key 58 is provided with a 20 , n ^ ^ ^ of the respective planes A through 

plane Aselectmg key 58a for selecting plane Am the left file c caD be d ako b d ^ tbe match ^ m the 

as images to be panned, a plane B selecting key 58b for match afea 3 b the operator 

selecting plane B in the middle file as images to be panned, , , ... , , , . , 

and a plane C selecting key 58c for selecting plane C in the In ^ w * ere ^ 15 ? an f d b * dragging, the 

right file as images to be panned. The CPU 27 recognizes the 25 operator drags the match image m the match area 3 of 

plane specified by the clicked one of the selecting keys 58a *siredp lane in the up.and-aowii 1 or r^t-and-left direction, 

through 58c as plane to be panned. ™ e C ™ 2 J recognizes the dragge^ plane, as.plane to be 

° u • u* j* »■ i en - panned, and controls display so that the match lmageTof the 

Then, when a right direction specifying key 59a is *\ \ . " ... •> -f r • , _ "--jr. jr^Cl , 

* j *u /^utt * 1 j« 1 Z « *L * u dragged plane: is panned -in the up- L andidownyor^ght=and- 

operated, the CPU 27 controls display so that the match _ . \. Kf * tL . , ■ ■■■■ „ 6 

. r . ' . . . - c . u 1 1 4 ju r*u 30 left direction^ After the operator performs an off-operation 

unage id the match area 3 of the plane selected by one of the mAiMir*', r . , ,. , t^y . Ai _ 

, . Co *u l eo ■ j • ,t • t, on the mousebutton, the CPU 27 controls display so that the 

selecting keys 58a through 58c is panned in the right _ . <^>ki r > . . . , , 

direction according to the operating time, and after the fasaxm of the P^^.also panned accordingly, 

panning of the match image, the CPU 27 controls display so M a result » the unages of the respective planes can be 

that fusion image is panned accordingly. panned to desired positions. 

In addition, when a left direction specifying key S9b is 35 Next, in the case where zoom rates of the images in the 

operated, the CPU 27 controls display so that the match 3 > 4 and 5 are adjusted, the operator clicks the zoorrT 

image in the match area 3 of the plane selected by one of the rate^changing mode specifying key 48 shown in FIG^WW 

selecting keys 58a through 58c is panned in the left direction th#mouse mode area 8. When the zoom rate changing mode 

according to the operating time, and after the panning of the specifying key 48 is clicked, the CPU 27 controls display of 

match image, the CPU 27 controls display so that fusion a zoom rate adjusting panel shown in FIG. 14 on the display 

image is panned accordingly. panel 49. 

In addition, when a high-speed right direction specifying The zoom rate adjusting panel is provided with a zoom 

key 59c is operated, the CPU 27 controls display so that the rate changing key 60 for changing zoom rates of images, a 

match image in the match area 3 of the plane selected by one 45 200111 ra te display input section 61 for displaying a current 

of the selecting keys 58a through 58c is panned in the right zoom rate and directly inputting a desired zoom rate through 

direction at high speed according to the operating time, and a key, and a fit key 62 for displaying images according to 

the CPU 27 controls display so that fusion image is panned z°°ni rates along sizes of the respective areas, 

according to the panning of the match image. When the images in one of the areas 3, 4 and 5 are selected 

In addition, when a high-speed left direction specifying 50 and a zoom-in key 60a of the zoom rate changing key 60 is 

key 59a* is operated, the CPU 27 controls display so that the clicked, the CPU 27 controls display so that the images in 

match image in the match area 3 of the plane selected by one the selected area are gradually zoomed in according to the 

of the selecting keys 58a through 58c is panned in the left operating time, and when a zoom-out key 60 is clicked, the 

direction at high speed according to the operating time, and CPU 27 controls display so that the images in the selected 

the CPU 27 controls display so that fusion image is panned 55 area are gradually zoomed out according to the operating 

according to the panning of the match image. time. Moreover, when a high-speed zoom- in key 60c of the 

In addition, when an up direction specifying key 59c is rate changing key 60 is clicked, the CPU 27 controls 
operated, the CPU 27 controls display so that the match display so that the images in the selected area are zoomed in 
image in the match area 3 of the plane selected by one of the at high speed according to the operating time, and when a 
selecting keys 58a through 58c is panned in the up direction 60 ni B h " s P eed zoom-out key 604 is clicked, the CPU 27 con- 
according to the operating time, and the CPU 27 controls trols display so that the images in the selected area are 
display so that fusion image is panned according to the zoomed out at high speed according to the operating time, 
panning of the match image. When the zoom rate of the images is changed, the CPU 27 

In addition, when a low direction specifying key 59/ is displays the current zoom rate of the images on the zoom 

operated, the CPU 27 controls display so that the match 65 rate display input section 61. 

image in the match area 3 of the plane selected by one of the As a result, the operator can adjust the zoom rates of the 

selecting keys 58a through 58c is panned in the low direc- images while recognizing the current zoom rates. 
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In addition, in the medical image processing apparatus of the left file in the areas 3 through 5. When the front display 

the present invention, the zoom rate of the images can be key 63 is clicked, the CPU 27 controls display of the images 

adjusted also by direct key input. In this case, the operator in the left file in the areas 3 through 5 to be displayed in the 

selects an image whose zoom rate is to be changed, and front direction, and when the rear display key 64 is clicked, 

inputs a desired zoom rate into the zoom rate display input 5 the CPU 27 controls display of the images in the left file in 

section 61 using the keyboard, etc. When the zoom rate is the areas 3 through 5 to be displayed in the rear direction, 

inputted, the CPU 27 controls display of the selected image in addition, as shown in FIG. 15B, the right side display 

according to the mputted^zo'omjrate^ key 65 and left side display key 66 specify the display 

In addition, the zoom rate of the images can be changed direction of the images in the middle file in the areas 3 

also by dragging by the operator. 10 through 5. When, the right side display key 65 is clicked, the 

In the case where the zooapte^ftheunages are changed CPU 27 controls display of the images in the middle file in 

by dragging, the operatormoves the cursor to a desired the areas 3 through 5 to be displayed in the right side 

image (base image, match image and fusion image), and in direction, and when the left side display key 66 is clicked, 

this state, drags the mouse in the right or left direction. The toe CPU 27 controls display of the images in the middle file 

CPU 27 recognizes the image currently indicated by the 15 in the areas 3 through 5 to be displayed in the left side 

cursor as an image whose zoom rate is to be changed, and direction. 

when image is dragged in the right direction, the CPU 27 In addition, as shown in FIG. 158, the head display key 

displays the image which has been enlarged according to the 67 and foot display key 68 specify the display direction of 

dragging amount, and when the image is dragged in the left the images in the right file in the areas 3 through 5. When the 

direction, the CPU 27 controls display of the image which 20 head display key 67 is clicked, the CPU 27 controls display 

has been zoomed out according to the dragging amount. of the images in the right file in the areas 3 through 5 to be 

Further, in the medical image processing apparatus of the displayed in the head-to-foot direction, and when the foot 

present invention, the images can be displayed with zoom display key 68 is clicked, the CPU 27 controls display of the 

rates according to sizes of the display regions 3a through 3c, images in the right file in the areas 3 through 5 to be 

4a through 4c and Sa through 5c of the images. In this case, 25 displayed in the foot-to head direction, 

the operator selects images in the desired display regions 3a The automatic control of the display directions of the 

through 3c, 4a through 4c and 5a through 5c and clicks a fit images are made based on the imaging position (imaging 

key 62. direction) and imaging pose which are added as header data 

When the fit key 62 is clicked, the CPU 27 controls to the coaxial tomogram data of the image data, 
display of the selected images having sizes according to the Namely, on the image storage section 23 shown in FIG. 3, 
display regions. As a result, the sizes of the display images as mentioned above, a three-dimensional pseudo image is 
can be adjusted to the most suitable sizes with one touch of formed and stored based on the imaging positions (imaging 
the fit key 62, and thus the convenience of the medical image directions) and imaging poses of the respective two- 
processing apparatus can be improved. ^ dimensional coaxial tomograms. For this reason, the CPU 27 

In the medical image processing apparatus of the present can previously recognize the imaging direction of the three- 
invention, since the zoom rate of the images can be adjusted, dimensional image formed on the image storage section 23, 
the base images and match images imaged by different and when the right side display key 65, for example, is 
modalities can be adjusted and composed so as to be clicked, the CPU 27 controls the image processing section 
displayed with the same size. For this reason, a diagnosis can ^ 22 so that image in the right side direction is formed from 
be made based on a fusion image which has been formed by the three-dimensional image based on the recognized imag- 
adjusting the comparing objects (base images and match ing direction. 

images) to the same size, and the accuracy of the diagnosis As a result, an image in a direction specified by the keys 

can be obtained. 63 through 68 can be displayed automatically. 

Next, the changes in the display directions and display 45 Here, in the case where the image data do not have data 

positions using the display direction changing mode sped- of the imaging positions (imaging directions) and imaging 

fying key 46 and display position changing mode specifying poses, the CPU 27 recognizes the direction of the coaxial 

key 47 can be applied to the subtle adjustment in the display tomogram as the head-to-foot direction, and controls display 

direction, but in the medical image processing apparatus, of the image in the specified direction accordingly, 

usually the images in the areas 3 through 5 can be displayed 50 Next, the display density of a fusion image, which has 

automatically in the display direction which is usually used. been formed based on the base image and match image 

Namely, the display direction is specified in the view whose display direction and display position and zoom rate 

mode area 9 shown in FIG. 15A in the operation panel are adjusted, can be adjusted by a density adjusting slider 70 

display area 2. provided to the fusion balance area 10 shown in FIG. 16 in 

The view mode area 9 is provided with display direction 55 me operation panel display area 2. 

specifying keys 69 which are composed of a front display In the case where the operator sees the fusion images and 

key 63 for specifying display of images in the front direction wants to enhance the match images (in the case where the 

(A), a rear display key 64 for specifying display of images operator wants to obtain fusion images where density of the 

in the rear direction (P), a right side display key 65 for match images has been increased), the operator moves the 

specifying display of images in the right side direction (R), eo density adjusting slider 70 in the righLdirection using the 

a left side display key 66 for specifying display of images in mouse. Moreover, in the case where the'Spfiato? wants to 

the left side direction (L), a head display key 67 for enhance the base images (in the case where the operator 

specifying display of images in the head-to-foot direction wants to obtain fusion images where density of the base 

(H), and a foot display key 68 for specifying display of images has been increased), the operator moves the density 

images in the foot-to -head direction (F). 65 adjusting slider 70 in the left direction using the mouse. 

As shown in FIG. 15B, the front display key 63 and rear In the initial setting state, the CPU 27 displays the 

display key 64 specify the display direction of the images in respective fusion images of the respective match images and 
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base images with the same density, and controls display of changes in cursor positions based on the cursor positions 

the density adjusting slider 70 on a center position of the inputted by the operator. 

move region. However, when the density adjusting slider 70 The cursor position changing key 77, the cursor position 

is moved in the right direction by the operator, the CPU 27 changing key 78 and the cursor position changing key 79 are 
forms and displays the fusion images where the density of 5 respectively provided correspondingly to the cross-sectional 

the match images has been increased according to the moved cursors Al and A2, the cross-sectional cursors Bl and B2, 

amount Moreover, when the density adjusting slider 70 is and the cross-sectional cursor C 

moved in the left direction, the CPU 27 detects it and forms a ^ changing key 77a of the cursor position 

and displays the fusion images where the density of the base changing key 77 is clicked by the operator, the CPU 27 

image has been increased according to the moved amount. 10 the movement of the cross-sectional cursor Al 

As a result, the operator can obtain the fusion images shown in FIG. 19 in the right direction according to the 
where desired images have been enhanced, and thus a operating time, and controls the movement of the cross- 
diagnosis of a complex portion, etc. can be simplified. sectional cursor A2 in the down direction. When a left 

Next, the position area 11 in the operation panel display changing key 77b is clicked, the CPU 27 controls movement 
area 2 is provided with a surface fit key 71 for specifying 15 of the cross-sectional cursor Al in the left direction and 
that, as shown in FIG. 17, the respective images in the base moves the cross-sectional cursor A2 in the up direction 
area 3 and match area 4 are aligned by setting the region of according to the operating time. When a quick right chang- 
interest (ROI), a point fit key 72 for specifying that the ing key 77c is clicked, the CPU 27 controls the quick 
respective images in the areas 3 and 4 are aligned by setting movement of the cross-sectional cursor Al in the right 
the fit points, and a set key 73 for specifying display of the 20 direction and controls the quick movement of the cross- 
dialogue box (FIG. 24) showing the rotation amount and sectional cursor A2 in the down direction according to the 
panning amount calculated on a basis of the original display operating time. When a quick left changing key 77 d is 
direction and display position of the images currently dis- clicked, the CPU 27 controls the quick movement of the 
played in the match area 4. cross-sectional cursor Al in the left direction and controls 

The keys 71 through 73 select "alignment in ROI" or 25 the quick movement the cross-sectional cursor A2 in the up 

"alignment at fit points" when the match images and base direction according to the operating time, 

images are aligned, and the details of them will be men- In addition, when a nght^c^ng^:keyl78a:of the cursor 

tioned later. position.changing keyjffl IJs^clicked^y;the:operator, the CPU 

Next, the save area 12 in the operation panel display area 3Q 27 controls r the moyemeht.of thejrgss^sectional cursors Bl 

2 is provided with a save key for specifying recording of and B2 in the ri^tllirectibrrshown in FIG719 according to 

images, when the save key is clicked by the operator, the the opera tmg^time. When~aieft~cbanging"keyi78!&iis clicked, 

CPU 27 controls storage of "the match images" displayed the CPU 27 contrpJte-the_mpypm^ 

currently in the match area 4 into the three-dimensional cursors Bl and-B2rin-me-left^ to the 

image data base 26 shown in FIG. 3. 35 operating timeTWhen~a~ quick right changing key 78c is 

As a result, the desired match images can be saved. clicked, the CPU 27 controls the quick movement of the 

Next, when the cross-sectional display mode (MPR dis- cross-sectional cursors Bl and B2 in the right direction 

play mode) is selected by the display mode switching key 41 according to the operating time. When a quick left changing 

mentioned on reference to FIG. 7, the CPU 27 controls key 7Sd is clicked, the CPU 27 controls the quick movement 

display of a coaxial tomogram and cross-sectional cursors ^ of tne cross-sectional cursors Bl and B2 in the left direction 

Al, A2, Bl, B2 and C shown in FIG. 19 for specifying according to the operating time. 

positions on the coaxial tomogram in the image display area In addition, whenjiar.down changinglssy^fl of the cursor 

1, and controls display of a cross-sectional cursor display/ position changmgkeyJ79-to^ the CPU 

non-display selecting key 74, a cursor moving condition 27 controls toe_moyem£ntTG7^ cursor C 

switching key 75, a cursor position display input section 76, 45 shown in HG^19Mn-4nej^ to the 

cursor position changing keys 77 through 79 shown in FIG. operating time^WhenTa-up-chaTaging key^7lft> is clicked, the 

18 in the grid area 13 in the operation panel display area 2. CPU 27 contrcris^e-movementof^ cursor 

The cross-sectional cursor display/non-display selecting C in the up dkectWaceord^ When 

key 74 is composed of a cross-sectional cursor display key a dow n changing key 79c is clicked, the CPU 27 

74a and a cross-sectional cursor non-display key 74b. When 50 controls the quick movement of the cross-sectional cursor C 

the cross-sectional cursor display key 74a is clicked, the m th e down direction according to the operating time. When 

CPU 27 controls display of the respective cross-sectional a ( l u ^ Si U P changing key 79d is clicked, the CPU 27 controls 

cursor Al, A2, Bl, B2 and C. Moreover, when the cross- the quick movement of the cross-sectional cursor C in the up 

sectional cursor non-display key 74b is clicked, the CPU 27 direction according to the operating time, 

does not display the cross-sectional cursors Al, A2, Bl, B2 55 By making it possible to display and to move the cross- 

and C. sectional cursors Al, A2, Bl, B2 and C, the operator can 

The cursor moving condition switching key 75 switches diagnose on a basis of a desired position on a coaxial 

between independent movements of the respective cross- tomogram, and thus the convenience of diagnosis can be 

sectional cursors Al, A2, Bl, B2 and C and simultaneous improved. 

(synchronous) movement of them. The CPU 27 switches eo The following describes an operation when in the medical 

these modes by the specifying by the mouse by the operator. image processing apparatus of the present invention, by 

The cursor/position display input section 76 displays setting fit points on the match images and base images and 

current positions of the respective cross-sectional cursors aligning the images, fusion images are formed. 

Al, A2, Bl, B2 and C, and is a section into which desired First, for example, in the case where an image imaged by 

cursor positions are directly inputted by the operator. The 65 the MRI apparatus is aligned with an image imaged by the 

CPU 27 displays cursor positions changed by cursor position X-ray CT apparatus so that the fusion image is displayed, the 

changing keys 77 through 79, mentioned later, and controls operator specifies it and capture the respective images. 
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More specifically, in the case where the respective images Next, when the adjustment for the base images and match 

to be compared or one of the images are stored in the images is completed, the operator selects a method of 

three-dimensional image data base 26, the CPU 27 controls aligning the images. In the medical image processing appa- 

reading-out from the three-dimensional imag;e-data base 26 ra tus of the present invention, the images are aligned by 

according to the specifying by the operatot^and captures the 5 setting the fit points or ROI. 

respective images or one of the images^In another way, in fi fa ^ ^ ^ ^ . m ^ d b ^ 

the case where the respective images to be compared or one ^ fit k ^ Qf ^ ^ ^ fi f£ ^ shoWQ 

T/rPi? 2 ^ * nG 17 * the P osition area 11 the P oinl fi^ey 72 

Sc?25 p g g p * clicked > the cpu 27 co » trols d *p la y *» » mode 

When the CPU 27 captures, the respective images, three, 10 JW?* 45 in the mouse^pde area 8 as shdwh m FIG. 

dimensional X-ray images^formedon the image storage " 80 that it shows the operator that the setting mode is for 

section 23 shown in FIG. 3 based orTa plurality of two- alignment. 

dimensional X-ray coaxial tomograms as mentioned above, In addition, at the same time, the CPU 27 displays the 

and they are "base images" as "images with which images dialogue box shown in FIG. 20 in the operation panel 

are aligned". Moreover, the CPU 27 forms the three- 15 display area 2. The dialogue box is provided with a fit point 

dimensional X-ray images on the image storage section 23 number display section 80 on which the numbers of the fit 

shown in FIG. 3 based on a plurality of two-dimensional P<» nt set to the base images are displayed, fit point coordi- 

MRI coaxial tomograms, and they are "match images" as Bate display sections 82 on which coordinates X, Y and Z of 

"images which are aligned with the base image" the fit points set to the baserimage^are^displayed, fit pointy 

Next, when the three-dimensional- X-ray -images of the 20 changing keys 81 for changing the fit points displayed on the 

respective apparatuses are formed on the image storage fit P°i?t coordinate.display sections 82, and a'delete key 83 

section 23, the CPU 27 controls display of the formed for canceling the fit points set to the base images displayed 

three-dimensional images on the image display area 1 of the 011 fit P oint number display section 80. 

image display section 21, and controls display of the opera- In addition, the dialogue box is provided with a fit point 

tion panel provided with the above-mentioned keys on the number display section 84 on which the numbers of the fit 

operation panel display area 2. points set to the match images are displayed, fit point 

Hie display formats of the images are selected by the coordinate display sections 86 on which coordinates X, Jf, 

1-section display key 31 and 9-section display key 32 and Z of the fit points set to the match images are displayed, 

described on reference with FIG. 4. When the one-section 30 fit P oint changing keys 85 for changing the fit points 

display key 31 is clicked, the CPU 27 displays one section displayed on the fit point coordinate display section 86, and 

of the image selected by the study selecting key 33 on the a dclete kcv 87 for canceling the fit points set to the match 

whole image display area 1, and when the 9-section display images displayed on the fit point number display section 84. 

key 32 is clicked, the CPU 27 divides the image display area In addition, the dialogue box is provided with a all delete 

1 into nine parts as mentioned on reference with FIG. 6 so 35 key 88 for canceling all the fit points set to the base images 

as to control display of the base image on the upper part, the and match images, a setting end key 89 which is clicked 

match image on the middle part and the fusion image on the when setting of the fit points is ended, an execution speci- 

lower part. fying key 90 for specifying start of the alignment at the fit 

The display modes of the respective images displayed as points, and a cancel key 91 which is clicked when the 

one or nine sections of the images is selected by the display 40 alignment at the fit points is stopped and another process is 

mode switching key 41 shown in FIG. 7, and in the initial executed. 

setting, the cross-section display mode (MPR display mode) For example, as shown in FIG. 1, when surfaces of a front 

is set. For this reason, when the operator does not specify the image, right side image, and image of a top of the head as 

switching of the display modes, the CPU 27 controls the the base images (images of the X-ray CT apparatus) are 

image processing section 22 so that the^oaxial tomograms 45 displayed in the regions 3a through 3c in the base area 3, and 

are formed from the mge -dimensional Images formed in the surfaces a front image, right side image and an image of a 

image storage section 23, and -displays 9 sections or 1 top of the head portion as the match images (images of the 

section of the coaxial tomograms formed in such a manner MRI apparatus) are displayed in the regions 4a through 4c 

shown in FIG. 21. of the match area 4, the operator, first, sets one fit point or 

In addition, when the operator clicks the display mode 50 a plurality of fit points to positions of the base images having 

switching key 41, the CPU 27 switches the display mode anatomic characteristics such as an eyeball and an ear hole, 

from the MPR display mode which is the initial setting state When the fit points are set, as shown by the fit points F1A 

to the surface display mode, and controls the image pro- through F3Ain FIG. 1, the CPU 27 marks the positions of 

cessing section 22 so that the surface image is formed from the base images, to which the fit points have been set, with 

the mrce-dimensional images formed on the image storage 55 white points, etc. Moreover, the positions are numbered in 

section 23. Moreover, the CPU 27 displays 9 sections or one the order that the fit points have been set, and each of the 

section of the surface image formed as shown in FIG. 1. numbers is displayed on the fit point number display section 

Next, when the 9 sections of the coaxial image or surface 80 forthe base images shown in FIG. 20, and the coordinates 

image are displayed in the image display area 1, the operator X, Yind.Z of the fit points corresponding to the displayed 

changes a window value, adjusts ^the-display direction, 60 number are displayed on the respective coordinate display 

display position and zoom rate using the bright" mode sections 82a, S2b and 82c. 

specifying key 44, display direction changing mode speci- Here, even if the fit points are set only on the front image, 

fying key 46, the display position changing mode specifying for example, the CPU 27 marks the corresponding positions 
key 47 and the zoom rate changing mode specifying key 48 "~ of the right side image and image of a top of the head 

described on reference with FIG. 10 and FIGS. 12 through 65 portion. 

14 so as to adjust the display direction, display position, In the case where the operator wants to confirm the 

zoom rate, etc. of the base images and match images. coordinates X, Y and Z of fit points other than the one 
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currently displayed, the operator clicks the display changing 
keys 81 shown in FIG. 20. Every time an up changing key 
81a of the display changing keys 81 is clicked, the CPU 27 
increments the number and displays it on the fit point 
number display section 80, and controls display of the 
coordinates X, Y and Z of the fit points corresponding to the 
displayed number on the respective coordinate display sec- 
tions 82fl, 82b and 82c. 

In addition, every time a down changing key 81b of the 
display changing keys 81 is clicked, the CPU 27 increments 
the number and displays it on the fit point number display 
section 80, and controls display of the coordinates X, Y and 
Z of the fit points corresponding to the displayed number on 
the respective coordinate display sections 82a, 82b and 82c. 

As a result, the operator can confirm the coordinates X, Y 
and Z of the set fit points. 

In addition, in the case where the operator wants to delete 
the fit points of the base images set in such a manner, the 
operator clicks the display changing keys 81 so that the 
number of the fit point to be deleted is displayed on the fit 
point number display section 80, and clicks the delete key 
83. When the delete key 83 is clicked, the CPU 27 deletes 
the number and coordinates X, Y and Z displayed on the fit 
point number display section 80. 

When the setting of the fit points of the base images is 
completed, the operator sets one fit point or a plurality of fit 
points on the position(s) of the match images which are the 
same as the position(s) of the base images. In this example, 
as shown in FIG. 1, three fit points FIB through F3B are set 
on the match images. 

When the fit points are set, the CPU 27 numbers the 
positions in the order that the fit points have been set, 
controls display of the numbers on the fit point number 
display section 84 for the match images shown in FIG. 20, 
and controls display of the coordinates X, Y and Z of the fit 
points corresponding to the displayed number on respective 
coordinate display sections 86a, 866 and 86c. 

In the case where the operator wants to confirm the 
coordinates X, Y and Z of fit points other than the fit point 
currently displayed, the operator clicks the display changing 
keys 85 shown in FIG. 20. 

Every time an up changing key 58a of the display 
changing keys 85 is clicked, the CPU 27 increments the 
number and displays it on the fit point number display 
section 84, and controls display of the coordinates X, Y and 
Z of the fit points corresponding to the displayed number on 
the respective coordinate display sections 86a, 86b and 86c. 

In addition, every time a down changing key 85b of the 
display changing keys 85 is clicked, the CPU 27 increments 
the number and displays it on the fit point number display 
section 84, and controls display of the coordinates X, Y and 
Z of the fit points corresponding to the displayed number on 
the respective coordinate display sections 86a, 86b and 86c. 

As a result, the operator can confirm the coordinates X, Y 
and Z of the set fit points. Moreover, As mentioned above, 
by clicking the display changing keys 81 for the base 
images, the coordinates X, Y and Z of the fit points set on 
the base images are displayed on the fit point coordinate 
display section 82. For this reason, by setting the fit points 
of the match images while watching the coordinates, the 
operator can set the fit points on the same positions of the 
base images and match images. 

Next, in the case where the operator wants to delete a fit 
point of the match images set in such a manner, the operator 
clicks the display changing keys 85 so that the number of the 
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fit point to be deleted is displayed on the fit point number 
display section 84, and clicks the delete key 87. When the 
delete key 87 is clicked, the CPU 27 deletes the number and 
coordinates X, Y and Z displayed on the fit point number 

5 display section 84. 

Here, in the case where after the fit point of the base 
images or the fit points of the match images is deleted, new 
fit point is set, the CPU 27 gives the number of the deleted 
fit point to the new fit point. 

10 Next, when the setting of the fit points arc ended, the 
operator clicks the setting end key 89 or execution specify- 
ing key 90. 

When the setting end key 89 or execution specifying key 
90 is clicked, the CPU 2 7 calculates a coor dinatejransfp r- 

1 mation matrix such that the positions^fWe^t - pSn1s"s8tion 
the base images coincide with positions of corresponding fit 
points set on the match images, and aligns the match images 
with the base images based on the coordinate transformation 
matrix and displays them, and composes both the aligned 
images so as to form fusion images. 

More specifically, the CPU 27 calculates the coordinate 
transformation matrix according to so-called "method of 
least square", namely, calculates a coordinate transformation 

^ matrix T according to the following equation (1) such that 
the sum of the square of a distance (/Pb.P'&a*) between a 
coordinate ?% Mi which is obtained by transforming the fit 
point (i-1 to n) of the match image by a prescribed 
transformation matrix T and the fit point P Bi (i=l to n) of the 

J0 corresponding base image. 

" (l) 
Cost(7) = £P^%^ 2 =>0 P% Mi = P„ r T 



This coordinate transformation matrix T is composed of a 
rotation component and a parallel component on the XYZ 
axes and can be represented by the following equation (2). 

T^cahIU{ayjfy(fi)^ (2) 

40 

Scale: zoom rate between base data and match data 
Rx(a): rotation matrix of an angle a about the X-axis 
Ry(p): rotation matrix of an angle p about the Y-axis 
45 Rz(y): rotation matrix of an angle 7 about the Z-axis 
Tx(tx): parallel panning matrix of a panning amount tx in 

the direction of the X-axis 
iy(ty): parallel panning matrix of a panning amount ty in 
the direction of the Y-axis 
so Tz(tz): parallel panning matrix of a panning amount tz in 
the direction of the Z-axis 
The CPU 27 displays the match image and base image 
with the match image being aligned with the base image 
based on the coordinate transformation matrix T calculated 
55 in such a manner. Then, the CPU 27 composes both the 
aligned images so as to form a fusion image, and controls 
display of the fusion image on the fusion area 5 in the image 
display area 1. 
As a result, as shown in FIG. 1, for example, a fusion 
60 image, which has been obtained by composing the surface 
image imaged by the X-ray CT apparatus as the base image 
with the surface image imaged by the MRI apparatus as the 
match image, can be displayed on the fusion area 5. 
For this reason, the images can be compared with each 
65 other realistically and visually, thereby greatly contributing 
to the accurate diagnosis and simplifying of a medical 
treatment plan by a doctor, etc. 
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The above explanation refers to the case where the surface 
images are selected as the base image and match image and 
they are aligned and composed at fit points, but the medical 
image processing apparatus of the present invention can also 
select coaxial tomograms as the base image and match 
image and align and compose them with each other at fit 
points so as to display the obtained image. 

In this case, the operator clicks the display mode switch- 
ing key 41 so as to select the MPR display mode. When the 
MPR display mode is selected, the CPU 27 controls the 
image processing section 22 so that a coaxial tomogram is 
formed from three-dimensional images formed in the image 
storage section 23, and controls displays of 9 sections or one 
section of the coaxial tomogram formed as shown in FIG. 
21. 

Next, in the case where 9 sections of the coaxial tomo- 
gram is displayed in the image display area 1, the operator 
adjusts a change in the window value, display direction, 
display position and zoom rate using the bright mode 
specifying key 44, display direction changing mode speci- 
fying key 46, display position changing mode specifying 
key 47 and zoom rate changing mode specifying key 48 
explained on reference with FIG. 10 and FIGS. 12 through 
14 so as to adjust the display direction, display position, 
zoom rate, etc. of the base image and match image. 

Next, the operator clicks the point fit key 72 shown in 
FIG. 17 so as to select the fit mode, and sets fit points FA1 
through FA4 and FBI through FB4 on the base images and 
match images as shown in FIG. 21, for example. As a result, 
numbers of the fit points and coordinates X, Y and Z are 
displayed on the dialogue box shown in FIG. 20. 

Next, when the setting of the fit points are ended, the 
operator clicks the setting end key 89 or execution specify- 
ing key 90 of the dialogue box shown in FIG. 20. 

When the setting end key 89 or execution specifying key 
90 is clicked, the CPU 27 calculates a coordinate transfor- 
mation matrix such that positions of the fit points set on the 
base images coincide with positions of corresponding fit 
points set on the match images based on the aforementioned 
equations (1) and (2), and displays the match images and 
base images with the match images being aligned with the 
base images using the coordinate transformation matrix. 
Then, by composing the aligned images, as shown in FIG. 
21, fusion images, which have been obtained by composing 
coaxial tomograms of the X-ray CT apparatus with coaxial 
tomograms of the MRI apparatus, are formed and displayed 
on the fusion area 5. 

As a result, in the similar manner to the above one, the 
images can be compared with each other realistically and 
visually, thereby contributing to the accurate diagnosis and 
simplifying of a medical treatment plan by a doctor etc. 

In the medical image processing apparatus of the present 
invention, in the case where the images of different modali- 
ties are aligned with each other and displayed, "reliability" 
of the alignment can be displayed. 

Namely, when the operator clicks the setting end key 89 
or execution specifying key 90 in the dialogue box shown in 
FIG. 20, the CPU 27 controls display of the fusion images 
of the base images and match images as mentioned above, 
and controls display of the dialogue box shown in FIG. 22 
in the operation panel display area 2. Then, the CPU 27 
controls display of the "reliability" such as "0.0001" in the 
dialogue box. 

The relationship between an actual shifting amount of the 
images and the reliability which is experimentally obtained 
is shown in the graph of FIG. 23. As is clear from the graph, 
the actual shifting amount of the images and the reliability 
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are interrelative. Therefore, as the reliability increases, the 
actual shifting amount of the images becomes larger, and as 
the reliability decreases, the actual shifting amount of the 
images becomes smaller. Namely, the actual shifting amount 

5 can be estimated by the reliability. 

For this reason, the operator can measure the trustwor- 
thiness of the fusion image by confirming the "reliability" 
displayed in the dialogue box after the images are aligned, 
and it can be useful for the diagnosis, medical treatment 

io plan, etc. 

Here, after the operator confirms the "reliability", the 
operator clicks an end key 103 provided to the dialogue box. 
As a result, the CPU 27 controls non-display of the dialogue 
box, and waits for a process to be specified thereafter. 

15 Next, the medical image processing apparatus of the 
present invention, a rotation amount and panning amount 
calculated on a-basis of4he original "di^lay^direction-and 
display position-ofthe match image can be confirmed, and 
in the case of this confirmation, the operator clicks the^sct^ 

20 key 73-shown in FIG. 17. > 

When the set key 73 is clicked, the CPU 27 displays a 
dialogue box shown in FIG. 24 on the operation panel 
display area 2. 
This dialogue box is provided with a rotation amount 

25 display/input section 97 for displaying a shift of an angle on 
a basis of the original position of the match image and 
inputting a desired angle by the operator thereon, a panning 
amount display/input section 98 for displaying a parallel 
panning amount on a basis of the original position of the 

30 match image and inputting a desired parallel panning 
amount by the operator thereon, and a default key 99 which 
is clicked when the match image is returned from a current 
position to the original position. 

In addition, the dialogue box is provided with a setting 

35 end key 100 which is clicked when the match image is 
rotated and panned parallel according to the values inputted 
into the rotation amount display/input section 97 and pan- 
ning amount display/input section 98 by the operator and the 
setting is ended, an execution key 101 which is clicked when 

40 the setting is not ended and the match image is rotated and 
panned parallel according to the values inputted into the 
rotation amount display/input section 97 and panning 
amount display/input section 98 by the operator, and a 
cancel key 102 which is clicked when the values inputted 

45 into the rotation amount display/input section 97 and pan- 
ning amount display/input section 98 are canceled. 

When the set key 73 is clicked, the CPU 27 controls 
display of the shift of the angle and parallel panning amount 
per coordinates X, Y and Z on the rotation amount display/ 

50 input section 97 and panning amount display/input section 
98 in the dialogue box. 

In the case where the displayed shift of the angle and 
parallel panning amount obtain desired values, the operator 
inputs the desired values into the rotation amount display/ 

55 input section 97 and panning-. amount display/input section 
98, and clicks the setting end key 100 or execution key 101. 
When the key 100 or key 101 is clicked, the CPU 27 rotates 
and parallel pans the match image according to the inputted 
values and controls display of the match image. As a result, 

60 the display position of the match image can be a desired 
display position. 

In the medical image processing apparatus of the present 
invention, the base image and match image can be aligned 
with each other by setting a region of interest (ROI) on the 

65 base image. 

In the case where the images are aligned with each other 
in the ROI, the operator, as mentioned above, displays 9 
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sections of the surface image of different modalities, and 
clicks the surface fit key 71 shown in FIG. 17. When the 
surface fit key 71 is clicked, the CPU 27 controls display of 
a dialogue box shown in FIG. 25 on the operation panel 
display area 2. 

This dialogue box is provided with ROI form selecting 
keys 92 for selecting a form of the ROI to be set, a delete key 
93 for deleting the set ROI, a setting end key 94 which is 
clicked when the setting of ROI is ended, a cancel key 95 
which is clicked when the alignment in the ROI is canceled, 
and an execution key 96 which is clicked when execution of 
the alignment in the set ROI is specified. 

Here, the setting end key 94 and execution key 96 specify 
starting of the execution of the alignment of the images, but 
when the setting end key 94 is clicked, the CPU 27 ends the 
setting mode of the ROI after the alignment of the images, 
and when the execution key 96 is clicked, the CPU 27 does 
not end the setting mode of the ROI even after the alignment 
of the images is ended. 

ROI is set on the base image, for example, and in the case 
where rectangular ROI is set, the operator clicks a rectan- 
gular ROI selecting key 92a of the ROI form selecting keys 
92. Moreover, in the case where circular ROI is set, the 
operator clicks a circular ROI selecting key 92b of the ROI 
form selecting keys 92. 

Next, the operator drags from a starting point to an end 
point of a portion of the base image to which ROI is set using 
the mouse. 

More specifically, in the case where the setting of rect- 
angular ROI is selected, the operator drags from the starting 
point 0 to the end point P of a portion to which ROI is set 
as shown by dotted line in FIG. 26 A using the mouse. When 
the dragging from the starting point O to the end point P is 
executed by the mouse, the CPU 27, as shown by a solid line 
in FIG. 26A, controls display of the rectangular ROI whose 
diagonal is the dragged line with, for example, red, and 
controls display of "knobs" on the four corners of the ROI. 

In addition, in the case where the setting of circular ROI 
is selected, the operator, as shown by a dotted line in FIG. 
26B, drags from the starting point O to the end point P of a 
portion to which ROI is set using the mouse. When the 
dragging from the starting point 0 to the end point P is 
executed by the mouse, the CPU 27 controls display of, as 
shown by a solid line in FIG. 26B, elliptic ROI with red, and 
controls display of "knobs" on the four corners which 
surround the elliptic ROI. 

As a result, as shown in FIG. 27, for example, the 
rectangular ROI is inconclusively set on the surface image 
of the base image. 

When the "knobs" are displayed, the set ROI can be 
deformed, so when the operator moves the knobs using the 
mouse, the CPU 27 controls display of the ROI deformed 
according to the moving amount. 

The ROI which has been inconclusively set in such a 
manner is defined by clicking the defining key of the mouse, 
and when the defining key is clicked, the CPU 27 controls 
display of the ROI, which has been displayed with red, with 
yellowish green, for example, and defines and sets the 
rectangular ROI shown in FIG. 27. When the display color 
of the ROI is changed into yellowish green, the operator can 
recognize the definition of the set ROL 

In the case where deformation, etc. of the ROI is edited 
again after the definition, the operator clicks the inside of the 
ROI. As a result, the CPU 27 controls display of the ROI, 
which has been displayed with yellowish green, with red 
again. As a result, the form of the ROI can be edited again 
by moving the "knobs" as mentioned above. 
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Here, since the alignment in ROI is executed according to 
the surface form of the image in the ROI, it is preferable that 
the size of ROI set by the operator is limited to not less than 
a suitable size such as not less than 30 pixels. 

5 In the case where the set ROI is deleted, the operator 
clicks the inside of the ROI. As a result, the CPU 27 controls 
display of the ROI, which has been displayed with yellowish 
green, with red again. 
Next, the operator confirms that the ROI is displayed with 

10 red and clicks the delete key 93. When the delete key 93 is 
clicked after clicking the inside of the ROI, the CPU 27 
deletes the ROI from the base image. As a result, unneces- 
sary ROI is deleted and resetting, etc. is possible. 
In such a manner, when ROI is set in a desired region of 

15 the base image and the setting end key 94 or execution key 
96 is clicked, the CPU 27 calculates a coordinate transfor- 
mation matrix based on the following equation such that a 
surface form in the ROI set on the base image can coincide 
with a surface form of the corresponding match image. 

20 As an average value or a square average value of a 
distance between a coordinate of the pixels on the surface in 
the ROI set on the base image and a surface coordinate of the 
nearest match image is smaller, the surface form in the ROI 
of the base image and the surface form of the corresponding 

22 match image coincide with each other more accurately. 
Therefore, the CPU 27 a coordinate transformation matrix 
Tby the aforementioned "method of least square" according 
to the following equation (3) or (4) such that the average 
value or square average value of a distance (JV B p% M ^ 

30 between one point P Bl (i-1 to n) in the ROI set on the base 
image and a coordinate P%Mi, which is obtained by trans- 
forming a point P^ (i-1 to n) of the match image which is 
the nearest to the point ? Si according to prescribed coordi- 
nate transformation T, becomes minimum. 

35 

1 « (3) 
Cost(7) = -Yp Bl P% Ml => P% Mi = P Mi ■ T 
n M 

Cost(7) = -Yp Bi P%Mt Z P% Mi = Pui T 

The coordinate transformation matrix T is also, as men- 
tioned above, composed of a rotation component on the 
45 respective XYZ axes and parallel component, and it can be 
represented by the following equation (5). 

TScale te{ayity®)MYYT<*>Ty&)'Tz(ty (5) 

The CPU 27 displays the base image and match image 
so with the match image being aligned with the base image 
based on the coordinate transformation matrix T calculated 
in such a manner. Then, the CPU 27 composes both the 
aligned images so as to form the fusion image, and controls 
display of the fusion image on the fusion area 5 in the image 
55 display area 1, and controls display of "reliability" on the 
dialogue box shown in FIG. 22. 

As a result, as shown in FIG. 27, for example, the fusion 
image, which is obtained by composing a surface image 
imaged by the X-ray CT apparatus as the base image with a 
60 surface image imaged by the MRI apparatus as the match 
image, can be displayed on the fusion area 5. 

For this reason, the images can be compared with each 
other realistically and visually, thereby contributing to the 
accurate diagnosis and simplifying of a medical treatment 
65 plan by a doctor, etc. 

Finally, in the above description of the embodiments, data 
for a plurality of coaxial tomograms are captured, but data 
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only for one coaxial tomogram may be captured. Id this case, 
it is difficult to display three-dimensional pseudo images, but 
the alignment at the aforementioned fit points and in the ROI 
can be executed without any problem, and it makes it 
possible to compare two-dimensional fusion images. 5 

In addition, the coordinate transformation matrix is cal- 
culated by the method of least square, but the optimizing 
methods, described later, such as " steepest descent method", 
"simulated annealing method" and "down hill simplex 
method" may be used. 10 

More specifically, in "the steepest descent method", an 
initial variable is given, a grade of its point is obtained, the 
minimum value is calculated while the variable is being 
moved in a direction where the grade becomes negative (a 
falling direction of a function), and the variable to be the 15 
minimum value is considered as the optimized variable. 
Moreover, in the "simulated annealing method", the variable 
is changed so that the value of the function becomes smaller 
gradually, and a variable in which the value of the function 
becomes the smallest is considered as the optimized vari- 20 
able. Moreover, in the "down hill simplex method", plural 
combinations of variables are set, mapping of the variables 
is moved geometrically towards the bottom of the function, 
and the optimized variable is finally obtained. 

The medical image processing apparatus of the present 25 
invention can superimpose and compose the images imaged 
by modalities using different imaging method on and with 
each other in the same position and with the same size, and 
display the fusion image. For this reason, the images can be 
compared with each other realistically and visually, thereby 30 
contributing to the accurate diagnosis and simplifying of a 
medical treatment plan by a doctor, etc. 

Here, the aforementioned embodiments are just examples 
of the present invention, and it will be obvious that the 
present invention may be varied in many ways according to 35 
its design, etc. as long as such variations are not to be 
regarded as a departure from the spirit and scope of the 
invention. 

What is claimed is: 

1. A medical image processing apparatus comprising 40 
image capturing means for capturing a plurality of medi- 
cal images obtained by imaging an object to be exam- 
ined; 

aligning means for aligning the medical images captured 

by said image capturing means; 
fusion image forming means for forming a fusion image 

by superimposing the medical images aligned by said 

aligning means on each other; 
display means for displaying the medical images thereon; 50 

and 

display control means for controlling the display of the 
respective medical images captured by said image 
capturing means and the fusion image of the respective 
medical images formed by said fusion image forming 55 
means on said display means. 

2. The medical image processing apparatus according to 
claim 1, further comprising: 

ROI setting means for setting desired regions of the 
respective medical images displayed on said display 60 
means or a desired region of one of the medical images 
as a region of interest, 

wherein said aligning means aligns the respective 
medical images with each other so that image infor- 
mation in the region of interest set by said ROI 65 
setting means coincides with each other for plural of 
said respective medical images. 
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3. The medical image processing apparatus according to 
claim 1, further comprising: 

fit point setting means for setting fit points to be reference 
points for the alignment on the respective medical 
images displayed on said display means, 
wherein said aligning means aligns the respective 
medical images with each other by superimposing 
the respective medical images so that the fit points 
set by said fit point setting means coincide with each 
other. 

4. The medical image processing apparatus according to 
claim 3, wherein: 

said display control means detects point coordinates of the 
fit points set by said fit point setting means so as to 
display the coordinates on said display means, the 
apparatus further comprising: 

point coordinate changing means for changing the point 
coordinates of the fit points displayed on said display 
means by inputting desired point coordinates, 
wherein when the point coordinates of the fit points 
are changed by said point coordinate changing 
means, said aligning means aligns the medical 
images based on the fit points whose point coor- 
dinates have been changed. 

5. The medical imago processing apparatus according to 
claim 1, wherein said image capturing means captures a 
plurality of coaxial tomograms imaged by different imaging 
methods. 

6. The medical image processing apparatus according to 
claim 5, further comprising: 

three-dimensional image forming means for forming 
three-dimensional pseudo images of the respective 
imaging methods based on said plurality of two- 
dimensional coaxial tomograms captured by said image 
capturing means, 

wherein said aligning means aligns the three- 
dimensional images formed by said three- 
dimensional image forming means, and said fusion 
image forming means superimposes the three- 
dimensional images aligned by said ali gnin g means 
on each other so as to form fusion images. 

7. The medical image processing apparatus according to 
claim 6, further comprising: 

ROI setting means for setting desired regions of the 
respective medical images displayed on said display 
means or a desired region of one of the medical images 
as a region of interest, 

wherein said aligning means aligns the medical images 
so that image information in the region(s) of interest 
set by said ROI setting means coincides with each 
other. 

8. The medical image processing apparatus according to 
claim 6, further comprising: 

fit point setting means for setting fit points to be reference 
points for the alignment on the respective medical 
images displayed on said display means, 
wherein said aligning means superimposes the respec- 
tive medical images so that the fit points set by said 
fit point setting means coincide with each other so as 
to align the medical images with each other. 

9. The medical image processing apparatus according to 
claim 8, wherein: 

said display control means detects point coordinates of the 
fit points set by said fit point setting means so as to 
display the coordinates on said display means, the 
apparatus further comprising: 



04/16/2004, EAST Version: 1.4.1 



5,954, 

29 

point coordinate changing means for changing the point 
coordinates of the fit points displayed on said display 
means by inputting desired point coordinates, 
wherein when the point coordinates of the fit points 
are changed by said point coordinate changing 5 
means, said aligning means aligns the respective 
medical images based on the fit points whose point 
coordinates have been changed. 

10. The medical image processing apparatus according to 
claim 1, further comprising display changing means for 10 
changing a display position and zoom rate when the respec- 
tive medical images displayed on said display means are 
two-dimensional images, or changing a display position, 
display direction and zoom rate when the respective medical 
images displayed on said display means are three- is 
dimensional images. 

11. The medical image processing apparatus according to 
claim 1, wherein when display of the fusion image and the 
respective medical images captured by sale image capturing 
means on said display means is controlled, said display 20 
control means controls simultaneous display of a plurality of 
images viewed from plural directions regarding each of the 
fusion image and the respective medical images. 
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12. A medical image processing apparatus comprising: 
image capturing means for capturing a plurality of medi- 
cal images obtained by imaging an object to be exam- 
ined; 

aligning means for aligning the medical images captured 

by said image capturing means; 
fusion image forming means for forming a fusion image 
by superimposing the medical images aligned by said 
aligning means on each other; 
display means for displaying the medical images thereon; 
and 

display control means for controlling the display of the 
fusion image of the respective medical images formed 
by said fusion image forming means on said display 
means, 

wherein when display of the fusion image on said 
display means is controlled, said display control 
means controls simultaneous display of a plurality of 
images viewed from plural directions. 

4 4*** 
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